White balance correction including indicative white color determination based on regions in a divided image

ABSTRACT

An entered image is divided into a plurality of blocks and it is determined, on a per-block basis, whether image data within an applicable block is indicative of the color white, based upon a condition, from among a plurality of conditions, that conforms to the position of each block in the image. A white balance correction is performed based upon data of a block determined to be indicative of the color white.

FIELD OF THE INVENTION

This invention relates to a white balance correction apparatus andmethod. More particularly, the invention relates to a white balancecorrection apparatus and method used in a device such as a digitalcamera or digital video camera.

BACKGROUND OF THE INVENTION

FIG. 13 is a block diagram illustrating the structure of a single CCDtype digital camera as one example of a conventional image sensingapparatus according. In FIG. 13, a solid-state image sensing device 1such as a CCD has its surface covered by an RGB color filter of, e.g., aBayer-type array, which enables color image sensing. The optical imageof an object that impinges upon the image sensing device 1 via a lens(not shown) is converted to an electric signal by the image sensingdevice 1. In order to eliminate noise from the electric signal obtainedby the conversion, the signal is processed by a CDS/AGC circuit 2, afterwhich a conversion is made to a digital signal by an A/D convertercircuit 3 pixel by pixel in successive fashion.

The digital signal that is output from the A/D converter circuit 3 hasits white gain adjusted by a white balance circuit 4, whence theresulting signal is sent to a luminance notch circuit 12. The luminancenotch circuit 12 uses a vertical low-pass filter (VLPF) to executeprocessing for reducing signal gain of a frequency in the vicinity ofthe Nyquist frequency in the vertical direction. Gain reductionprocessing by a horizontal low-pass filter (HLPF) is executed similarlyin the horizontal direction. Such a filter shall be referred to as a“luminance notch filter” below. Next, a horizontal band-pass filter(HBPF) circuit 13 and vertical band-pass filter (VBPF) circuit 16 raisethe frequency, which is slightly lower than the Nyquist frequencyweakened by the notch filters.

Amplitude is adjusted subsequently by PP (aperture peak) gain circuits14 and 17 in both the horizontal and vertical directions, and then lowamplitude is cut, thereby eliminating noise, by base clipping (BC)circuits 15 and 18. The horizontal and vertical components aresubsequently added by an adder 19, main gain is applied by an APC(Aperture Control) main gain circuit 20, and the resultant signal isadded to a baseband signal by an adder 21. A gamma conversion circuit 22then performs a gamma conversion and a luminance correction (YCOMP)circuit 23 executes a luminance-signal level correction based uponcolor.

Next, as an example of color signal processing, a color interpolationcircuit 5 executes an interpolation with regard to all pixels in such amanner that all color pixel values will be present, and a colorconversion matrix (MTX) circuit 6 converts each of the color signals toa luminance signal (Y) and color difference signals (Cr, Cb).Color-difference gain of low- and high-luminance regions is thensuppressed by a chroma suppression (CSUP) circuit 7, and band is limitedby a chroma low-pass filter (CLPF) circuit 8. The band-limited chromasignal is converted to an RGB signal and is simultaneously subjected toa gamma conversion by a gamma conversion circuit 9. The RGB signalresulting from the gamma conversion is again converted to Y, Cr, Cbsignals, gain is adjusted again by a chroma gain knee (CGain Knee)circuit 10, and a linear clip matrix (LCMTX) circuit 11 makes a minorcorrection of hue and corrects a shift in hue caused by individualdifferences between image sensing devices.

Processing executed by the white balance circuit 4 in the image sensingapparatus of FIG. 13 will now be described. The image signal output fromthe image sensing device 1 and converted to a digital signal by the A/Dconverter circuit 3 is divided into a plurality (any number of) blocksof the kind shown in FIG. 14, and color evaluation values Cx, Cy, Y arecalculated for each block based upon the following equations:Cx=(R−B)/YCy=(R+B−2G)/YY=(R+G+B)/2  (1)

The color evaluation values Cx, Cy of each block calculated inaccordance with Equations (1) are compared with a previously set whitedetection region (described later). If the evaluation values fall withinthe white detection region, it is assumed that the block is white andthen the summation values (SumR, SumG, SumB) of respective ones of thecolor pixels of the blocks assumed to be white are calculated. Whitebalance gains kWB_R, kWB_G, kWB_B for each of the colors RGB are thencalculated from the summation values using the following equations:kWB _(—) R=1.0/SumRkWB _(—) G=1.0/SumGkWB _(—) B=1.0/SumB  (2)

The white balance circuit 4 performs the white balance correction usingthe white balance gains thus obtained.

FIGS. 15A and 15B are graphs illustrating a white detection region 101.In order to obtain the white detection region 101, a white object suchas a standard white sheet (not shown) is sensed from high to low colortemperatures using light sources at arbitrary color temperatureintervals, and the color evaluation values Cx, Cy are calculated basedupon Equations (1) using the signal values obtained from the imagesensing device 1. Next, Cx and Cy obtained with regard to each of thelight sources are plotted along the X axis and Y axis, respectively, andthe plotted points are connected by straight lines. Alternatively,plotted points are approximated using a plurality of straight lines. Asa result, a white detection axis 102 from high to low color temperaturesis produced. In actuality, there are slight variations in spectraldiffraction even for the color white. For this reason, the whitedetection axis 102 is provided with some width along the direction ofthe Y axis. This region is defined as the white detection region 101.

The conventional white balance detection apparatus, however, has certaindrawbacks. For example, consider a case where a close-up of the humanface is taken in the presence of a light source such as sunlight havinga high color temperature. Though the color evaluation values of a whitesubject in the presence of sunlight are distributed as indicated by area103 in FIG. 15A, the color evaluation values of the human complexion aredistributed as indicated by area 105 in FIG. 15A. These values aredistributed in an area substantially the same as that (the areaindicated at 104 in FIG. 15A) of color evaluation values of the colorwhite photographed in the presence of a light source such as whitetungsten having a low color temperature.

Further, the color evaluation values are distributed as indicated inarea 106 in FIG. 15B in a case where an area (on the order of 7000 K) inwhich the blue of the sky has grown pale, as at the horizon or at theboundaries of clouds, is included in a scene in, say, a scenery mode.This substantially coincides with the distribution of evaluation values(area 107 in FIG. 15B) of the color white sensed under cloudy conditionsor in the shade. As a consequence, the color temperature of the scene isjudged to be higher than it actually is (color temperature under clearskies is on the order of 5500 K) and the pale blue of the sky iscorrected to white. This represents a judgment error.

SUMMARY OF THE INVENTION

The present invention has been made in consideration of the abovesituation and has as its object to reduce erroneous decisions regardingwhite evaluation, thereby performing a better white balance correction.

According to the present invention, the foregoing object is attained byproviding a white balance correction apparatus comprising; a dividingunit adapted to divide an image into a plurality of areas; a whiteevaluation unit adapted to determine whether image data within everydivided area is indicative of the color white according to the positionof each divided area in the image; and a white balance correction unitadapted to perform a white balance correction based upon image datadetermined to be indicative of the color white.

According to the present invention, the foregoing object is alsoattained by providing a white balance correction method comprising;dividing an image into a plurality of areas; determining whether imagedata within every divided area is indicative of the color whiteaccording to the position of each divided area in the image; andperforming a white balance correction based upon image data determinedto be indicative of the color white.

Other features and advantages of the present invention will be apparentfrom the following description taken in conjunction with theaccompanying drawings, in which like reference characters designate thesame or similar parts throughout the figures thereof.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate an embodiment of the inventionand, together with the description, serve to explain the principles ofthe invention.

FIG. 1 is a block diagram illustrating a structure of a white balancecorrection apparatus according to a first embodiment of the presentinvention;

FIGS. 2A and 2B are diagrams illustrating white detection regionsaccording to the first embodiment of the present invention;

FIGS. 3A and 3B are diagrams illustrating the division of a screen inthe first embodiment of the present invention;

FIG. 4 is a flowchart illustrating white evaluation processing accordingto the first embodiment of the present invention;

FIG. 5 is a flowchart illustrating processing for determining whether ornot the color of complexion is present in an automatic mode according tothe first embodiment of the present invention;

FIGS. 6A and 6B are diagrams illustrating the division of a screen in afirst modification of the invention;

FIGS. 7A and 7B are diagrams illustrating white detection regionsaccording to the first modification of the present invention;

FIG. 8 is a diagram illustrating the division of a screen in a secondmodification of the invention;

FIGS. 9A and 9B are diagrams illustrating white detection regionsaccording to the second modification of the present invention;

FIGS. 10A and 10B are diagrams illustrating white detection regionsaccording to the second modification of the present invention;

FIG. 11 is a block diagram illustrating the structure of a white balancecorrection apparatus according to the second embodiment of the presentinvention;

FIGS. 12A to 12C are diagrams illustrating the division of a screenaccording to the second embodiment of the present invention;

FIG. 13 is a block diagram illustrating the structure of a conventionalimage sensing apparatus;

FIG. 14 is a diagram illustrating division of a screen indicating unitsfor performing white evaluation;

FIGS. 15A and 15B are diagrams illustrating a conventional whitedetection region; and

FIGS. 16A to 16C are diagrams illustrating examples of detectionpatterns for detecting a white region on a screen conforming tobrightness of an object in the first modification of the presentinvention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Preferred embodiments of the present invention will be described indetail in accordance with the accompanying drawings.

First Embodiment

FIG. 1 is a block diagram illustrating the structure of a white balancecorrection apparatus according to a first embodiment of the presentinvention. The white balance correction apparatus according to thisembodiment is capable of being used instead of the white balance circuit4 shown in FIG. 13, by way of example. The components of the imagesensing apparatus with the exception of the white balance circuit 4 willbe described below with reference to FIG. 13.

As shown in FIG. 1, the white balance correction apparatus includes amode determination unit 30 for determining the operating mode of theimage sensing apparatus (modes such as an automatic mode, portrait modefor sensing a person or persons, scenery mode for sensing scenery andmanual mode for allowing the user to set a white balance correctionvalue); an evaluation block dividing unit 31 for dividing the outputsignal of the image sensing device 1 into a plurality of evaluationblocks of the kind shown in FIG. 14; a white detection region storageunit 32 for storing a white detection region (referred to as a “standardwhite correction region” below) serving as a standard; a white detectionregion varying unit 33 for changing the white detection region whenappropriate using limit values; a pattern storage unit 34 for storing,classified by mode, patterns which are combinations of position on thescreen and a white decision region that has been changed by the whitedetection region varying unit 33 used in order to subject the evaluationblock at this position to white evaluation; a white evaluation unit 35for determining whether each evaluation block obtained by division bythe evaluation block dividing unit 31 is white or not; a WB coefficientcalculation unit 36 for calculating white balance (WB) coefficients,which are used in a WB correction, from signal values of evaluationblocks judged to be white by the white evaluation unit 35; a WBcoefficient storage unit 37 for storing WB coefficients obtained by theWB coefficient calculation unit 36; and a WB correction unit 38 forapplying a WB correction to the output signal of the image sensingdevice 1 using the WB coefficients stored in the WB coefficient storageunit 37. It should be noted that the white detection region storage unit32, pattern storage unit 34 and WB coefficient storage unit 37 may beimplemented by a single memory or by a plurality of memories.

The reference white detection region stored in the white detectionregion storage unit 32 will now be described. The description willrelate to a case where a primary-color filter is used as the imagesensing device 1.

In FIGS. 2A and 2B, reference numerals 201 and 202 denote a whitedetection region and a white detection axis, respectively. As in theexample of the prior art, a white object such as a standard white sheet(not shown) is sensed from high to low color temperatures using lightsources at arbitrary color temperature intervals, and the colorevaluation values Cx, Cy are calculated based upon Equations (1) usingthe signal values obtained from the image sensing device 1. Next, Cx andCy obtained with regard to each of the light sources are plotted alongthe X axis and Y axis, respectively, and the plotted points areconnected by straight lines or the plotted points are approximated by aplurality of straight lines. As a result, the white detection axis 202from high to low color temperatures is produced. It should be noted thatthe X axis corresponds to the color temperature of the light source andthe Y axis corresponds to the amount of correction in the greendirection (that is, the color-temperature direction of luminance and thecolor-temperature direction of fluorescent light). In actuality, thereare slight variations in spectral diffraction even for the color white.For this reason, the white detection axis is provided with some widthalong the direction of the Y axis. This region is defined as the whitedetection region 201. The data of the white detection region thusdefined is stored in the white detection region storage unit 32 when thewhite balance correction apparatus is manufactured or shipped.

Conventionally, the evaluation block dividing unit 31 divides the outputsignal of the image sensing device 1 into a plurality of evaluationblocks of the kind shown in FIG. 14, and the white evaluation unit 35calculates the color evaluation values Cx, Cy block by block using theEquations (1) and determines that a block is white if the calculatedcolor evaluation values Cx, Cy fall within the white detection region201. According to a characterizing feature of the first embodiment ofthe present invention, however, the size (limits along the direction ofcolor temperature in an example shown in FIGS. 2A and 2B) of the whitedetection region 201 can be changed by the white detection regionvarying unit 33 in accordance with relative position of the evaluationblock on the screen.

In FIG. 2A, a white detection region 203 is obtained by setting whitedetection limits Ll1, Lh1 with respect to the white detection region201, thereby applying limits in such a manner that the range of Cx ismade Ll1 to Lh1. A white detection region 204 in FIG. 2B is obtained bysetting white detection limits Ll2, Lh2 with respect to the whitedetection region 201, thereby applying limits that make the range of Cxequal to Ll2 to Lh2 of the white detection region 204 so as to cut thelow region of color temperatures from the white detection region 203.

The white evaluation unit 35 subjects the evaluation blocks to a whiteevaluation and the WB coefficient calculation unit 36 calculates the WBcoefficients (white balance gains) from the summed values of the pixelvalues in evaluation blocks determined to be white and stores thesecoefficients in the WB coefficient storage unit 37. The WB correctionunit 38 subjects the input image to a WB correction using thecoefficients stored in the WB coefficient storage unit 37.

White evaluation processing will now be described with reference toFIGS. 2A to 4B.

FIG. 3A illustrates a pattern setting example of a pattern stored in thepattern storage unit 34 in the automatic mode, and FIG. 3B illustrates apattern setting example of a pattern stored in the pattern storage unit34 in the portrait mode. Each pattern indicates a combination of theposition of each evaluation block and the size of the changed whitedetection region used when subjecting the evaluation block at thisposition to white evaluation. It may be so arranged that even though thepatterns are stored beforehand at the time of manufacture or shipping ofthe white balance correction apparatus, the user can alter the regionsettings.

First, at step S11 in FIG. 4, the mode determination unit 30 determineswhether the automatic mode or the portrait mode has been set. If theautomatic mode has been set, control proceeds to step S12, at which thewhite evaluation unit 35 acquires the region data of the pattern shownin FIG. 3A from the pattern storage unit 34. If the portrait mode hasbeen set, control proceeds to step S13, at which the white evaluationunit 35 acquires the region data of the pattern shown in FIG. 3B fromthe pattern storage unit 34.

Next, at step S14, the white evaluation unit 35 determines whether eachevaluation block is in region (1) or region (2). If it is in region (1)(“YES” at step S14), control proceeds to step S15, at which the colorevaluation values of the evaluation block are compared with the whitedetection region 203, which is shown in FIG. 2A, limited by the whitedetection region varying unit 33.

If the evaluation block is in region (2) (“NO” at step S14), controlproceeds to step S16, at which the color evaluation values of theevaluation block are compared with the white detection region 204, whichis shown in FIG. 2B, limited by the white detection region varying unit33. There is a high likelihood that a human face will be included in thecentral area of the screen. Accordingly, the limitation on the side oflow color temperature is set to be higher for the central area than forthe periphery of the screen, thereby applying a limitation to the whitedetection region 204 in such a manner that a complexion tone will not beevaluated incorrectly as being white.

If the color evaluation values of an evaluation block are found to fallwithin the white detection region 203 or 204 at either step S15 or S16,then control proceeds to step S17, at which the white evaluation unit 35decides that this evaluation block is white. If the color evaluationvalues of an evaluation block are found not to fall within the whitedetection region 203 or 204, then control proceeds to step S18, at whichthe white evaluation unit 35 decides that this evaluation block is notwhite.

An evaluation block thus determined to be white has its pixel valuessummed in order to calculate the white gains (WB coefficients), asdescribed above.

Whether the decision as to whether an evaluation block is white has beenrendered with regard to all evaluation blocks is determined at step S19.Steps S14 to S18 are repeated until all of the evaluation blocks havebeen evaluated.

Experiments have shown that excellent results can be obtained if thewhite detection limit Ll2 on the side of low color temperature is fixedat about 5000 K. However, it goes without saying that the presentinvention is not limited to 5000 K and that this can be changed asappropriate.

In accordance with the first embodiment, as described above, erroneousdecisions regarding white evaluation can be reduced by using a whitedetection region that differs depending upon position on the screen. Asa result, it is possible to perform a better white balance correction.

Further, by enlarging the area of the central region (2), as shown inFIG. 3B, in a case where the image sensing mode of the camera is madethe portrait mode, erroneous decisions regarding white evaluationascribable to complexion can be reduced.

However, though it may be considered that there is a high likelihoodthat complexion will be present at the central part of the screen if themode is the portrait mode, it may also be considered that there are manyinstances where complexion is not present at the central part of thescreen if the mode is the automatic mode. If the above-described settingof the screen and setting of white limits is made in a case wherecompletion is not present at the central part of the screen, the resultof color-temperature detection at the center of the screen will not fallbelow the white detection limit Ll2 (e.g., 5000 K) on the side of lowcolor temperature. A problem that arises as a consequence is that theresult of color temperature based upon an image captured under, e.g.,the A light source will become higher than the actual light-source colortemperature. Accordingly, a white balance correction of higher precisioncan be achieved by performing the determination regarding the absence orpresence of complexion before the operation indicated at step S12 inFIG. 4 and performing the operations from step S12 onward if complexionis judged to be present. This operation will be described with referenceto the flowchart of FIG. 5.

Evaluation blocks in which image data is determined to be white (theseblocks shall be referred to as “white evaluation blocks” below) aredetected at step S21 using the same white detection region (a regiondelimited by the white detection limits or a region in which thecolor-temperature region of complexion is contained in a white detectionregion that is not limited; e.g., either the white detection region 201or 203) with regard to all evaluation blocks in the central portion ofthe screen [region (2) in FIG. 3A] and peripheral portion [region (1) inFIG. 3A], light-source color temperature CtAround is obtained at stepS22 from data that is the result of summing and averaging the image dataof the white evaluation blocks in the peripheral portion of the screen,and light-source color temperature CtCenter is obtained at step S23 fromdata that is the result of summing and averaging the image data of thewhite evaluation blocks in the central portion of the screen. It shouldbe noted that the order of the processing steps S22 and S23 may bereversed or the processing of steps S22 and S23 may be executed inparallel.

Next, CtAround and CtCenter are compared at step S24. If the colortemperature CtCenter obtained from the central portion of the screen isless than the color temperature CtAround obtained from the peripheralportion of the screen, then it is judged at step S25 that there is ahigh likelihood that complexion is present in the central portion of thescreen. In other words, if

CtCenter<CtAround

holds, then the central portion of the screen is judged to containcomplexion and the light-source color temperature is calculated at stepS26 by performing the white evaluation in the automatic mode shown inFIGS. 2A to 4B.

On the other hand, if the color temperature CtCenter obtained from thecentral portion of the screen is substantially equal to or greater thanthe color temperature CtAround obtained from the peripheral portion ofthe screen, it is judged at step S27 that there is a high likelihoodthat complexion is not present at the central portion of the screen. Inother words, if

CtCenter≧CtAround

holds, then it is judged that complexion is not present, all evaluationblocks are compared with a common white detection region (step S28),white evaluation blocks are detected and the light-source colortemperature obtained is adopted.

Adding on the above processing makes it possible to reduce even furthererroneous decisions regarding white evaluation and makes it possible toperform a better white balance correction.

If the manual mode is found to be in effect at the mode discriminationstep S11, then, in a manner similar to that where absence of complexionis determined, all evaluation blocks are compared with a common whitedetection region, white evaluation blocks are detected and thelight-source color temperature obtained from image data in the whiteevaluation blocks is adopted.

(First Modification)

FIGS. 6A and 6B illustrate pattern setting examples for suppressingerroneous decisions that blue sky is the color white. FIG. 6Aillustrates an example of a pattern in the case of the automatic mode,and FIG. 6B illustrates an example of a pattern in the case of thescenery mode. In a manner similar to that of the first embodiment,whether an evaluation block is white or not is determined by comparingregions (1) and (2) with white detection regions limited using differentwhite detection limits.

Color evaluation values of an evaluation block in an image region inwhich the sky appears close to thin clouds or the horizon have adistribution substantially the same as that of color evaluation valuesof white points in the shade, as mentioned earlier. Consequently, anevaluation block in a portion of an image that is the sky is judgederroneously to be white. In other words, the sky is judged erroneouslyto be white, which has a high color temperature. Accordingly, whiteevaluation is carried out using different white detection regions in theupper and lower parts of the screen, as shown in FIG. 6A, with alimitation being applied by the white detection region varying unit 33using a different white detection limit on the side of high colortemperature.

In the first modification, as shown in FIG. 7B, a white detection limitLh4 on the side of high color temperature limiting the white detectionregion for the purpose of judging evaluation blocks at the upper portionof the screen is set to the side of low color temperature in comparisonwith a white detection limit Lh3 on the side of high color temperatureshown in FIG. 7A for the purpose of judging evaluation blocks at thelower portion of the screen, thereby arranging is so that pale blue willnot be judged erroneously as white.

Experiments have shown that excellent results can be obtained if thewhite detection limit Lh4 is fixed at about 5500 K. However, it goeswithout saying that the present invention is not limited to 5500 K andthat this can be changed as appropriate.

In accordance with the first modification, as described above, erroneousdecisions regarding white evaluation can be reduced by using a whitedetection region that differs depending upon position on the screen. Asa result, it is possible to perform a better white balance correction.

Further, by enlarging the area of the upper region (2), as shown in FIG.6B, in a case where the photographic mode of the camera is made thescenery mode, erroneous decisions regarding white evaluation at portionsof blue sky can be reduced.

In the first modification, brightness Bv of a subject may be detectedfrom the captured image data and white detection patterns of the kindshown in FIGS. 6A and 6B may be changed in accordance with brightness.For example, when Bv is greater than a preset value Bv2, the probabilitythat the image was captured outdoors is high (the percentage of areaoccupied by the sky is large). Accordingly, the proportion of evaluationblocks (2) in the upper part of the screen in which the white detectionrange is limited by a white detection region 209 shown in FIG. 10A isenlarged, as shown in FIG. 16B. On the other hand, when Bv is less thana preset value Bv1 (<Bv2), the probability that the image was capturedindoors is high, and therefore the proportion of evaluation blocks (2)in the upper part of the screen is reduced, as shown in FIG. 16A. If thebrightness of the subject is between Bv1 and Bv2, the proportion ofevaluation blocks (2) in the upper part of the screen in which the whitedetection range is limited by the white detection region 209 is decidedby linear calculation at Bv as indicated by the graph of FIG. 16C. Aneven more suitable white balance correction can be performed byexecuting such processing.

(Second Modification)

FIG. 8 illustrates an example of pattern setting for suppressingerroneous decisions relating to both complexion and the sky. Excellentresults are obtained if, by way of example, the following settings aremade for white evaluation of evaluation blocks in region (1) of FIG. 8:

white detection limit Lh5 on the side of high color temperature: 5500 K;

white detection limit Ll5 on the side of low color temperature:brightness variable (FIG. 9A);

the following settings are made with respect to the region (2) in FIG.8:

white detection limit Lh6 on the side of high color temperature:brightness variable;

white detection limit Ll6 on the side of low color temperature:brightness variable (FIG. 9B);

the following settings are made with respect to the region (3) in FIG.8:

white detection limit Lh7 on the side of high color temperature: 5500 K;

white detection limit Ll7 on the side of low color temperature: 5000 K(FIG. 10A); and

the following settings are made with respect to the region (4) in FIG.8:

white detection limit Lh8 on the side of high color temperature:brightness variable;

white detection limit Ll8 on the side of low color temperature: 5000 K(FIG. 10B).

It should be noted that the specific values indicated as the whitedetection limits Lh5, Lh7, Ll7 Ll8 are examples only. The presentinvention is not limited to these values and the values can be changesas appropriate.

In accordance with the second modification, as described above, use ismade of patterns in which the area of the screen is more finely dividedand white evaluation is performed using white detection regions thatdiffer depending upon position on the screen. Accordingly, a whitebalance correction of even higher precision can be achieved.

It should be noted that the pattern used can be changed depending uponthe image sensing mode in a manner similar to that of the firstembodiment.

Second Embodiment

FIG. 11 is a block diagram illustrating the brief structure of a whitebalance correction apparatus according to a second embodiment of thepresent invention. This embodiment differs from that of FIG. 1 in thatthe mode determination unit 30 in FIG. 1 is replaced by avertical/horizontal orientation determination unit 40 in FIG. 11 fordiscriminating horizontal and vertical orientations at the time of imagesensing. Components in FIG. 11 identical with those shown in FIG. 1 aredesignated by like reference numerals, and explanation of them areomitted.

The characterizing feature of the second embodiment is that thevertical/horizontal orientation determination unit 40 discriminateshorizontal and vertical orientations (rotation clockwise by 90° androtation counter-clockwise by 90°) based upon the output of a gravitysensor (not shown) and makes it possible to change the setting of whitedetection limits depending upon the state of photography.

FIGS. 12A to 12C are diagrams illustrating pattern setting examples.FIG. 12A illustrates an example of a pattern in a case where horizontalorientation has been determined by the vertical/horizontal orientationdetermination unit 40, FIG. 12B an example of a pattern in a case whererotation counter-clockwise by 90° has been determined by thevertical/horizontal orientation determination unit 40, and FIG. 12C anexample of a pattern in a case where rotation clockwise by 90° has beendetermined by the vertical/horizontal orientation determination unit 40.Point A in FIGS. 12A to 12C indicates the same corner and is shown inorder to facilitate an understanding of direction of rotation.

The evaluation blocks in the regions (1) to (4) of FIGS. 12A to 13C arejudged using, e.g., the white detection regions 207 to 210 shown inFIGS. 9A, 9B, 10A and 10B.

In accordance with the second embodiment, as described above, whiteevaluation of each evaluation block can be performed appropriatelyirrespective of camera orientation (horizontal or vertical) at the timeof image sensing.

It should be noted that the pattern settings shown in FIGS. 12A to 12Care one example. It may be so arranged that the pattern settings shownin FIGS. 3A, 3B and FIGS. 6A, 6B or other settings may be used, and thesettings may be changed depending upon the image sensing mode.

In the above embodiments, a case in which the white detection region islimited by the white detection region varying unit 33 has beendescribed. However, it may be so arranged that a plurality of whitedetection regions are stored in advance and any of the stored whitedetection regions is used in dependence upon the position of theevaluation block.

Other Embodiments

Software and hardware implementations of the above embodiments can besubstituted as appropriate.

Further, the above embodiments or the technical elements thereof may becombined as necessary.

Furthermore, even if all or part of the structure of the claims or ofthe embodiments forms a single apparatus, or even if a connection ismade to an image sensing apparatus such as a digital camera or videocamera or to another apparatus such as an image processing apparatusthat processes signals obtained from an image sensing apparatus, thepresent invention may serve as an element that constructs suchapparatus.

Further, the object of the present invention can also be achieved byproviding a storage medium storing program codes for performing theaforesaid processes to a computer system or apparatus (e.g., a personalcomputer), reading the program codes, by a CPU or MPU of the computersystem or apparatus, from the storage medium, then executing theprogram.

In this case, the program codes read from the storage medium realize thefunctions according to the embodiments, and the storage medium storingthe program codes constitutes the invention.

Further, the storage medium, such as a floppy disk, a hard disk, anoptical disk, a magneto-optical disk, CD-ROM, CD-R, a magnetic tape, anon-volatile type memory card, and ROM, and computer network, such asLAN (local area network) and WAN (wide area network), can be used forproviding the program codes.

Furthermore, besides aforesaid functions according to the aboveembodiments are realized by executing the program codes which are readby a computer, the present invention includes a case where an OS(operating system) or the like working on the computer performs a partor entire processes in accordance with designations of the program codesand realizes functions according to the above embodiments.

Furthermore, the present invention also includes a case where, after theprogram codes read from the storage medium are written in a functionexpansion card which is inserted into the computer or in a memoryprovided in a function expansion unit which is connected to thecomputer, CPU or the like contained in the function expansion card orunit performs a part or entire process in accordance with designationsof the program codes and realizes functions of the above embodiments.

In a case where the present invention is applied to the aforesaidstorage medium, the storage medium stores program codes corresponding tothe flowcharts shown in FIG. 4 and/or FIG. 5 described in theembodiments.

The present invention is not limited to the above embodiments andvarious changes and modifications can be made within the spirit andscope of the present invention. Therefore to apprise the public of thescope of the present inventions the following claims are made.

1. A white balance correction apparatus, comprising: a dividing unitadapted to divide an image into a plurality of areas; a pattern storageunit adapted to store patterns indicating combinations of positions ofeach of the divided areas and conditions used, the patterns including apattern that correlates the central portion of an image with a firstcondition and areas other than the central portion with a secondcondition; a white evaluation unit adapted to determine whether imagedata within every divided area is indicative of the color whiteaccording to the position of each divided area in the image, whereinwhite evaluation processing of all divided areas is executed using anidentical condition, a first color temperature calculated from thecentral portion of the image and a second color temperature calculatedfrom a peripheral portion of the image are compared, and said whiteevaluation unit is controlled so as to perform the white evaluationusing an identical condition, irrespective of the position of a dividedarea in the image, if the first color temperature is equal to or higherthan the second color temperature; and a white balance correction unitadapted to perform a white balance correction based upon image datadetermined to be indicative of the color white.
 2. The apparatusaccording to claim 1, further comprising a calculation unit adapted tocalculate a plurality of color evaluation values for every divided area.3. An image sensing apparatus equipped with a white balance correctionapparatus as set forth in claim
 1. 4. The apparatus according to claim3, further comprising an image sensing mode setting unit adapted todesignate the image sensing mode.
 5. A white balance correctionapparatus, comprising: a dividing unit adapted to divide an image into aplurality of areas; a pattern storage unit adapted to store patternsindicating combinations of positions of each of the divided areas andconditions used, the patterns including a pattern that correlates theupper portion of an image with a third condition and areas other thanthe upper portion with a fourth condition, and wherein white evaluationprocessing of all divided areas is executed using an identicalcondition, a first color temperature calculated from the upper portionof the image and a second color temperature calculated from a lowerportion of the image are compared, and said white evaluation unit iscontrolled so as to perform the white evaluation using an identicalcondition, irrespective of the position of a divided area in the image,if the second color temperature is equal to or higher than the firstcolor temperature; a white evaluation unit adapted to determine whetherimage data within every divided area is indicative of the color whiteaccording to the position of each divided area in the image; and a whitebalance correction unit adapted to perform a white balance correctionbased upon image data determined to be indicative of the color white. 6.The apparatus according to claim 5, further comprising a calculationunit adapted to calculate a plurality of color evaluation values forevery divided area.
 7. An image sensing apparatus equipped with a whitebalance correction apparatus as set forth in claim
 5. 8. The apparatusaccording to claim 7, further comprising an image sensing mode settingunit adapted to designate the image sensing mode.
 9. A white balancecorrection apparatus, comprising: a dividing unit adapted to divide animage into a plurality of areas; a pattern storage unit adapted to storepatterns indicating combinations of positions of each of the dividedareas and conditions used, the patterns including a pattern thatcorrelates the upper portion of an image with a third condition andareas other than the upper portion with a fourth condition, and whereinthe patterns further include a pattern which, if the image sensing modeis a scenery mode, correlates the upper portion of the image to thethird condition upon making the upper portion wider than when othermodes are in effect; a white evaluation unit adapted to determinewhether image data within every divided area is indicative of the colorwhite according to the position of each divided area in the image; and awhite balance correction unit adapted to perform a white balancecorrection based upon image data determined to be indicative of thecolor white.
 10. The apparatus according to claim 9, further comprisinga calculation unit adapted to calculate a plurality of color evaluationvalues for every divided area.
 11. An image sensing apparatus equippedwith a white balance correction apparatus as set forth in claim
 9. 12.The apparatus according to claim 11, further comprising an image sensingmode setting unit adapted to designate the image sensing mode.
 13. Awhite balance correction apparatus, comprising: a dividing unit adaptedto divide an image into a plurality of areas; a pattern storage unitadapted to store patterns indicating combinations of positions of eachof the divided areas and conditions used, wherein the patterns include apattern that correlates the upper part of the central portion of animage with a fifth condition, the lower part of the central portion witha sixth condition, the upper part of areas other than the centralportion with a seventh condition, and all other areas with an eighthcondition; a white evaluation unit adapted to determine whether imagedata within every divided area is indicative of the color whiteaccording to the position of each divided area in the image; and a whitebalance correction unit adapted to perform a white balance correctionbased upon image data determined to be indicative of the color white.14. The apparatus according to claim 13, wherein the conditions definediscrimination of the color white in image data based upon acolor-temperature region, limit values on the side of low colortemperature of the fifth and sixth conditions are set to be higher thanlimit values on the side of low color temperature of the seventh andeighth conditions, a limit value on the side of high color temperatureof the fifth condition is set to be lower than a limit value on the sideof high color temperature of the sixth condition, and a limit value onthe side of high color temperature of the seventh condition is set to belower than a limit value on the side of high color temperature of theeighth condition.
 15. The apparatus according to claim 13, furthercomprising a calculation unit adapted to calculate a plurality of colorevaluation values for every divided area.
 16. An image sensing apparatusequipped with a white balance correction apparatus as set forth in claim13.
 17. The apparatus according to claim 16, further comprising an imagesensing mode setting unit adapted to designate the image sensing mode.